Enhanced vehicle operation

ABSTRACT

A system includes a computer including a processor and a memory, the memory storing instructions executable by the processor to generate a synthetic image by adjusting respective color values of one or more pixels of a reference image based on a specified meteorological optical range from a vehicle sensor to simulated fog, and input the synthetic image to a machine learning program to train the machine learning program to identify a meteorological optical range from the vehicle sensor to actual fog.

BACKGROUND

Vehicles can be equipped with computing devices, networks, sensors andcontrollers to acquire data regarding the vehicle's environment and tooperate the vehicle based on the data. Vehicle sensors can provide dataconcerning routes to be traveled and objects to be avoided in thevehicle's environment. Operation of the vehicle can rely upon acquiringaccurate and timely data regarding objects in a vehicle's environmentwhile the vehicle is being operated on a roadway. Vehicles may usecomputing devices configured to identify objects from image datacollected by the vehicle sensors.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an example system for operating a vehicle.

FIGS. 2A-2C are example images with simulated fog.

FIG. 3 is a diagram of an example machine learning program.

FIG. 4 is a diagram of an example process for training a machinelearning program.

FIG. 5 is a diagram of an example process for operating the vehicle witha machine learning program.

DETAILED DESCRIPTION

A system includes a computer including a processor and a memory, thememory storing instructions executable by the processor to generate asynthetic image by adjusting respective color values of one or morepixels of a reference image based on a specified meteorological opticalrange from a vehicle sensor to simulated fog and input the syntheticimage to a machine learning program to train the machine learningprogram to identify a meteorological optical range from the vehiclesensor to actual fog.

The instructions can further include instructions to collect an imagewith the vehicle sensor, input the image to the machine learningprogram, output, from the machine learning program, the meteorologicaloptical range from the vehicle sensor to the actual fog, and actuate oneor more vehicle components based on the meteorological optical rangefrom the vehicle sensor to the actual fog.

The instructions can further include instructions to transition one ofthe one or more vehicle components from autonomous operation to manualoperation when the meteorological optical range from the vehicle sensorto the actual fog is below a distance threshold.

The instructions can further include instructions to transition avehicle from an autonomous mode to a manual mode when the meteorologicaloptical range from the vehicle sensor to the actual fog is below asecond distance threshold.

The instructions can further include instructions to actuate a firstcomponent when the meteorological optical range from the vehicle sensorto the actual fog is above a first threshold and to actuate a secondcomponent when the meteorological optical range from the vehicle sensorto the actual fog is above a second threshold.

The instructions can further include instructions to output, from themachine learning program, an object in the collected image and toactuate the one or more vehicle components based on the object output bythe machine learning program.

The instructions can further include instructions to identify an ambientlight of the reference image and to adjust the respective color valuesof the one or more pixels based on the ambient light.

The instructions can further include instructions to annotate thesynthetic image with the meteorological optical range from the sensor tothe simulated fog.

The instructions can further include instructions to adjust the colorvalues of one of the pixels in the reference image by decreasing thecolor values of the one of the pixels based on a transmissioncoefficient of light through fog and increasing an ambient light of thepixel based on the transmission coefficient.

The instructions can further include instructions to identify an initialdistance from the vehicle sensor to an object in one of the pixels ofthe reference image and to adjust the color values of the one of thepixels based on the initial distance and the specified meteorologicaloptical range from the vehicle sensor to the simulated fog.

The instructions can further include instructions to identify respectivemeteorological optical ranges from the vehicle sensor to the actual fogfor all pixels in an image collected by the vehicle sensor and toactuate one or more vehicle components based on a minimum meteorologicaloptical range from the vehicle sensor to the actual fog.

The instructions can further include instructions to identify an objectwith a second vehicle sensor and a distance from the second vehiclesensor to the object, and to suppress actuation of one or more vehiclecomponents based on the identified object when the distance from thesecond vehicle sensor to the object exceeds the meteorological opticalrange from the vehicle sensor to the actual fog.

The meteorological optical range is a distance at which a luminosity ofa beam of light extending from the vehicle sensor falls below aluminosity threshold.

A method includes generating a synthetic image by adjusting respectivecolor values of one or more pixels of a reference image based on aspecified meteorological optical range from a vehicle sensor tosimulated fog, the meteorological optical range being a distance atwhich a luminosity of a beam of light extending from the vehicle sensorfalls below a luminosity threshold and inputting the synthetic image toa machine learning program to train the machine learning program toidentify a meteorological optical range from the vehicle sensor toactual fog

The method can further include collecting an image with the vehiclesensor, inputting the image to the machine learning program, outputting,from the machine learning program, the meteorological optical range fromthe vehicle sensor to the actual fog, and actuating one or more vehiclecomponents based on the meteorological optical range from the vehiclesensor to the actual fog.

The method can further include transitioning one of the one or morevehicle components from autonomous operation to manual operation whenthe meteorological optical range from the vehicle sensor to the actualfog is below a distance threshold.

The method can further include transitioning a vehicle from anautonomous mode to a manual mode when the meteorological optical rangefrom the vehicle sensor to the actual fog is below a second distancethreshold.

The method can further include actuating a first component when themeteorological optical range from the vehicle sensor to the actual fogis above a first threshold and actuating a second component when themeteorological optical range from the vehicle sensor to the actual fogis above a second threshold.

The method can further include outputting, from the machine learningprogram, an object in the collected image and actuating the one or morevehicle components based on the object output by the machine learningprogram.

The method can further include identifying an ambient light of thereference image and adjusting the respective color values of the one ormore pixels based on the ambient light.

The method can further include annotating the synthetic image with themeteorological optical range from the sensor to the simulated fog.

The method can further include adjusting the color values of one of thepixels in the reference image by decreasing the color values of the oneof the pixels based on a transmission coefficient of light through fogand increasing an ambient light of the pixel based on the transmissioncoefficient.

The method can further include identifying an initial distance from thevehicle sensor to an object in one of the pixels of the reference imageand adjusting the color values of the one of the pixels based on theinitial distance and the specified meteorological optical range from thevehicle sensor to the simulated fog.

The method can further include identifying respective meteorologicaloptical ranges from the vehicle sensor to the actual fog for all pixelsin an image collected by the vehicle sensor and actuating one or morevehicle components based on a minimum meteorological optical range fromthe vehicle sensor to the actual fog.

The method can further include identifying an object with a secondvehicle sensor and a distance from the second vehicle sensor to theobject, and suppressing actuation of one or more vehicle componentsbased on the identified object when the distance from the second vehiclesensor to the object exceeds the meteorological optical range from thevehicle sensor to the actual fog.

A system includes a vehicle sensor, means for generating a syntheticimage by adjusting respective color values of one or more pixels of areference image based on a specified meteorological optical range fromthe vehicle sensor to simulated fog, the meteorological optical rangebeing a distance at which a luminosity of a beam of light extending fromthe vehicle sensor falls below a luminosity threshold, means forinputting the synthetic image to a machine learning program to train themachine learning program to identify a meteorological optical range fromthe vehicle sensor to actual fog, means for collecting an image with thevehicle sensor, means for inputting the image to the machine learningprogram, means for outputting, from the machine learning program, themeteorological optical range from the vehicle sensor to the actual fog,and means for actuating one or more vehicle components based on themeteorological optical range from the vehicle sensor to the actual fog.

The system can further include means for transitioning one of the one ormore vehicle components from autonomous operation to manual operationwhen the meteorological optical range from the vehicle sensor to theactual fog is below a distance threshold.

The system can further include means for outputting, from the machinelearning program, an object in the collected image and means foractuating the one or more vehicle components based on the object outputby the machine learning program.

Further disclosed is a computing device programmed to execute any of theabove method steps. Yet further disclosed is a vehicle comprising thecomputing device. Yet further disclosed is a computer program product,comprising a computer readable medium storing instructions executable bya computer processor, to execute any of the above method steps.

Fog can occlude images collected by vehicle sensors. That is, fogbetween an object and a vehicle sensor can obscure the object in theimage collected by the vehicle sensor. Identifying a distance to theobject from the vehicle sensor can be difficult in an image obscured byfog. A vehicle computer can operate a vehicle based on data from thevehicle sensor. When the data is occluded by fog, the vehicle computercan ignore data collected beyond a “meteorological optical range,” i.e.,a distance at which light attenuates to below a predetermined threshold,as described below. That is, the electromagnetic waves received by thesensor from beyond the meteorological optical range may be attenuated orreduced by the fog, and these attenuated waves may not accurately orprecisely provide data about the object. Using a machine learningprogram can identify the meteorological optical range of fog in an imagecollected by the vehicle sensor more quickly than, e.g., athree-dimensional depth detection algorithm. The quicker identificationof the meteorological optical range can improve operation of the vehiclecomputer by updating the distance beyond which the vehicle computershould ignore data as the vehicle travels along a route.

FIG. 1 illustrates an example system 100 for operating a vehicle 101. Acomputer 105 in the vehicle 101 is programmed to receive collected datafrom one or more sensors 110. For example, vehicle 101 data may includea location of the vehicle 101, data about an environment around avehicle, data about an object outside the vehicle such as anothervehicle, etc. A vehicle 101 location is typically provided in aconventional form, e.g., geo-coordinates such as latitude and longitudecoordinates obtained via a navigation system that uses the GlobalPositioning System (GPS). Further examples of data can includemeasurements of vehicle 101 systems and components, e.g., a vehicle 101velocity, a vehicle 101 trajectory, etc.

The computer 105 is generally programmed for communications on a vehicle101 network, e.g., including a conventional vehicle 101 communicationsbus such as a CAN bus, LIN bus, etc., and or other wired and/or wirelesstechnologies, e.g., Ethernet, WIFI, etc. Via the network, bus, and/orother wired or wireless mechanisms (e.g., a wired or wireless local areanetwork in the vehicle 101), the computer 105 may transmit messages tovarious devices in a vehicle 101 and/or receive messages from thevarious devices, e.g., controllers, actuators, sensors, etc., includingsensors 110. Alternatively or additionally, in cases where the computer105 actually comprises multiple devices, the vehicle network may be usedfor communications between devices represented as the computer 105 inthis disclosure. In addition, the computer 105 may be programmed forcommunicating with the network 120, which, as described below, mayinclude various wired and/or wireless networking technologies, e.g.,cellular, Bluetooth®, Bluetooth® Low Energy (BLE), wired and/or wirelesspacket networks, etc.

The memory can be of any type, e.g., hard disk drives, solid statedrives, servers, or any volatile or non-volatile media. The memory canstore the collected data sent from the sensors 110. The memory can be aseparate device from the computer 105, and the computer 105 can retrieveinformation stored by the memory via a network in the vehicle 101, e.g.,over a CAN bus, a wireless network, etc. Alternatively or additionally,the memory can be part of the computer 105, e.g., as a memory of thecomputer 105.

Sensors 110 can include a variety of devices. For example, variouscontrollers in a vehicle 101 may operate as sensors 110 to provide datavia the vehicle 101 network or bus, e.g., data relating to vehiclespeed, acceleration, position, subsystem and/or component status, etc.Further, other sensors 110 could include cameras, motion detectors,etc., i.e., sensors 110 to provide data for evaluating a position of acomponent, evaluating a slope of a roadway, etc. The sensors 110 could,without limitation, also include short range radar, long range radar,LIDAR, and/or ultrasonic transducers.

Collected data can include a variety of data collected in a vehicle 101.Examples of collected data are provided above, and moreover, data aregenerally collected using one or more sensors 110, and may additionallyinclude data calculated therefrom in the computer 105, and/or at theserver 125. In general, collected data may include any data that may begathered by the sensors 110 and/or computed from such data.

The vehicle 101 can include a plurality of vehicle components 115. Inthis context, each vehicle component 115 includes one or more hardwarecomponents adapted to perform a mechanical function or operation—such asmoving the vehicle 101, slowing or stopping the vehicle 101, steeringthe vehicle 101, etc. Non-limiting examples of components 115 include apropulsion component (that includes, e.g., an internal combustion engineand/or an electric motor, etc.), a transmission component, a steeringcomponent (e.g., that may include one or more of a steering wheel, asteering rack, etc.), a brake component, a park assist component, anadaptive cruise control component, an adaptive steering component, amovable seat, and the like.

For purposes of this disclosure, the vehicle 101 can operate in one of afully autonomous mode, a semi-autonomous mode, or a non-autonomous mode.A fully autonomous mode is defined as one in which each of vehicle 101propulsion (typically via a powertrain including an electric motorand/or internal combustion engine), braking, and steering are controlledby the computer 105. A semi-autonomous mode is one in which at least oneof vehicle 101 propulsion (typically via a powertrain including anelectric motor and/or internal combustion engine), braking, and steeringare controlled at least partly by the computer 105 as opposed to a humanoperator. In a non-autonomous mode, i.e., a manual mode, the vehicle 101propulsion, braking, and steering are controlled by the human operator.

The system 100 can further include a network 120 connected to a server125. The computer 105 can further be programmed to communicate with oneor more remote sites such as the server 125, via the network 120, suchremote site possibly including a processor and a memory. The network 120represents one or more mechanisms by which a vehicle computer 105 maycommunicate with a remote server 125. Accordingly, the network 120 canbe one or more of various wired or wireless communication mechanisms,including any desired combination of wired (e.g., cable and fiber)and/or wireless (e.g., cellular, wireless, satellite, microwave, andradio frequency) communication mechanisms and any desired networktopology (or topologies when multiple communication mechanisms areutilized). Exemplary communication networks include wirelesscommunication networks (e.g., using Bluetooth®, Bluetooth® Low Energy(BLE), IEEE 802.11, vehicle-to-vehicle (V2V) such as Dedicated ShortRange Communications (DSRC), etc.), local area networks (LAN) and/orwide area networks (WAN), including the Internet, providing datacommunication services.

FIGS. 2A-2C illustrate images 200 with simulated fog 205. In the contextof this document, “fog” is a visible aerosol containing water dropletsor ice crystals suspended in air above a roadway that can reduce orblock light transmission therethrough. Further in the context of thisdocument, “simulated fog” is a specification of color values of theimage 200, typically changing original color values, to mimic theocclusion caused by actual fog. That is, the image 200 can be matrix,each element of the matrix being a set of red-blue-green (RGB) colorvalues or a black-and-white (BW) color value. Each element of the matrixis a “pixel” of the image 200. Each pixel can have a set of colorvalues, i.e., numbers representing respective color components such asred-blue-green, that specify a color for the pixel. The simulated fog205 can be a set of pixels in which color values are adjusted to becloser to a white color than the original RGB or BW values. That is, inRGB, a pixel is white when its RGB values are 256-256-256. A pixel ofthe simulated fog 205 can have RBG color values that are between anoriginal set of RGB values of the pixel and the white color 256-256-256.FIG. 2A illustrates a reference image 200 with no simulated fog 205.FIG. 2B illustrates the reference image 200 with simulated fog 205 at adistance r₁ from the sensor 110, as described below. FIG. 2C illustratesthe reference image 200 with simulated fog 205 at a distance r₂ from thesensor 110, as described below. The “reference” image 200 can be animage 200 used to train a machine learning program, as described below.

The server 125 can apply the simulated fog 205 to the reference image200 by applying a color changing model to the image 200:F(x)=t(x)R(x)+L(1−t(x))  (1)where x is a pixel of the image 200, F(x) is a matrix representing theimage 200 with simulated fog 205, R(x) is a matrix representing theoriginal image 200, t(x) is a transmission coefficient, as describedbelow, and L is an ambient light, as described below.

The transmission coefficient t(x) is a number between 0 and 1 thatrepresents an amount of light that reaches the camera 110:t(x)=exp(−β−s(x))  (2)where β is an attenuation coefficient and s(x) is an initial distancefrom the camera 110 to an object in the reference image 200 without fog.The attenuation coefficient β is a value quantifying the exponentialdecay of transmission of light to the initial distance s(x) to theobject. Specifically, the attenuation coefficient β is based on ameteorological optical range (MOR) r. The meteorological optical range ris a distance at which a luminosity of a beam of light extending from anemitter (e.g., a light source at the sensor 110) falls below aluminosity threshold. The luminosity threshold is a predetermined valueset by a standard setting organization, e.g., the International CivilAviation Organization. For example, the luminosity threshold can be 5%.Thus, the server 125 can simulate fog 205 at specified MOR values totrain the machine learning program. The attenuation coefficient β canthus be defined according to the luminosity threshold, for example:β=2.996/r when the luminosity threshold is 5%.

The color changing model can use ambient light L(1−t(x)) to apply thesimulated fog 205. In this context, “ambient light” is a constant RBGvalue representing atmospheric light, e.g., a white color with an RBGvalue of 256-256-256. The ambient light L is thus applied to thereference image 200 to occlude objects in the reference image 200. Thatis, the ambient light L changes the color values of pixels xproportional to 1−t(x), generating the simulated fog 205. That is, asthe transmission coefficient t(x) increases, the amount of ambient lightL added to the pixel x reduces, reducing the amount of simulated fog 205in the pixel x.

The server 125 can apply the simulated fog 205 at specified MOR rvalues. That is, the server 125 can, for each pixel x in the image 200,adjust the color values of the pixel x according to Equations 1-2 above.For example, the server 125 can apply the simulated fog 205 at an MOR rvalue of r₁ to generate the simulated fog 205 of FIG. 2B. In anotherexample, the server 125 can apply the simulated fog 205 at an MOR rvalue of r₂ to generate the simulated fog 205 of FIG. 2C. Alternativelyor additionally, the server 125 can apply the simulated fog 205 at adifferent MOR r value to generate simulated fog 205 at the MOR r.

The server 125 can generate a synthetic image by adjusting respectivecolor values of one or more pixels of a reference image 200 based on aspecified MOR r from the sensor 110 to the simulated fog 205. A“synthetic image” is an image in which one or more pixels are adjustedby a computer program. For example, the synthetic image can be an image200 collected by a camera 110 in which simulated fog 205 is added. Thatis, the color changing model can output a synthetic image F(x), asdescribed above. The server 125 can generate a plurality of syntheticimages, each synthetic image including simulated fog 205 at a specifiedMOR r.

The server 125 can input a plurality of synthetic images to a machinelearning program. The server 125 can train the machine learning programto identify a MOR r from the vehicle sensor 101 to actual fog with theinput synthetic images. As described below, the server 125 can annotatethe synthetic images with the MOR r used to input the simulated fog 205.The server 125 can determined a difference between an output MOR r ofthe machine learning program and the annotation of the MOR r in thesynthetic image. The server 125 can input the difference to a costfunction (e.g., a least-squares equation). The server 125 can, usingtechniques such as back-propagation and gradient descent, adjust outputof the machine learning program until the cost function is minimized.

The machine learning program can output an object in the collected image200. That is, the machine learning program can be trained to identifyone or more objects in the image 200. The machine learning program canbe trained with images 200 with annotations of objects in the image 200.That is, using a cost function as described above between outputidentification of objects and the annotations in the images 200, theserver 125 can adjust output of the machine learning program to minimizethe cost function. The computer 105 can actuate one or more vehiclecomponents 115 based on the object output by the machine learningprogram.

The computer 105 can determine a meteorological optical range r ofactual fog in an image 200. The computer 105 can collect an image 200with a sensor 110, e.g., a camera. The computer 105 can input the imageto the machine learning program, as described above. The machinelearning program can output the MOR r from the sensor 110 to the actualfog for each pixel of the image 200. The computer 105 can identify aminimum MOR r that is a smallest MOR r value of the MOR r values of thepixels in the image 200.

The computer 105 can actuate one or more vehicle components 115 based onthe minimum MOR r from the sensor 110 to the actual fog. The MOR r is adistance beyond which the computer 105 can ignore data from one or moresensors 110 as unreliable compared to data collected from sensors 110without fog, i.e., where data from the sensors 110 in the absence of fogcould be reliable. That is, the data collected by the sensors 110 caninclude a distance at which the data were collected, e.g., a distance toan identified object. If the distance at which the data were collectedexceeds the identified MOR r, the computer 105 can ignore the data,suppressing use of the data to actuate components 115. For example, ifthe computer 105 receives data indicating that a roadway beyond the MORr is free of objects, the computer 105 can ignore the data and collectadditional data of the roadway when the vehicle 101 moves within the MORr of the roadway. Thus, the computer 105 can actuate components 115 onlywith data collected within the MOR r.

Each component 115 can have a minimum distance from the vehicle 101 atwhich the sensors 110 collect data for autonomous operation. That is,each component 115 can use data from the sensors 110, and each component115 can have a minimum distance from the vehicle 101 from which data iscollected to operate without manual input. For example, a brake can havea minimum distance of 60 meters because, upon actuating the brake, adistance to slow a vehicle 101 from a speed of 100 kilometers per hourto a stop is about 55 meters. Thus, the computer 105 can determine that,when the MOR r is less than 60 meters, the brake should be controlled bymanual input. In another example, a steering can have a minimum distanceof 30 meters.

The computer 105 can transition one or more vehicle components 115 fromfull or semi autonomous operation to manual operation when the MOR r isbelow a distance threshold. The distance threshold can be the minimumdistance for autonomous operation, as described above. That is, one ormore components 115 may require manual input when the MOR r is below thedistance threshold, and the computer 105 can transition the one or morecomponents 115 to accept the manual input. As the MOR r to the actualfog increases beyond the respective minimum distances of the components115, the computer 105 can actuate the components 115.

Additionally or alternatively, the computer 105 can transition thevehicle 101 from an autonomous mode to one of a semiautonomous mode or amanual mode, as described above, based on the MOR r. That is, ratherthan transitioning vehicle components 115 to manual operation one a timebased on the MOR r, the computer 105 can transition the vehicle 101 tothe semiautonomous mode or the manual mode, thereby transitioning one ormore vehicle components 115 to manual operation that could still operateautonomously at the identified MOR r. When the MOR r from the sensor 110to the actual fog is below a second distance threshold, the computer 105can transition the vehicle from the autonomous mode to the manual mode.The second distance threshold can be a largest minimum distance, asdescribed above, at which one of the components 115 can operateautonomously with data from the sensors 110. Because the minimumdistance of the brake is greater than the minimum distance of thesteering, the computer 105 can define the second threshold as theminimum distance of the brake and can determine to transition thevehicle 101 to the manual mode when the MOR r is less than the secondthreshold.

FIG. 3 is a diagram of an example machine learning program 300. Themachine learning program 300 can be a deep neural network (DNN) 300 thatcould be trained to identify a meteorological optical range r of actualfog in an image 200. The DNN 300 can be a software program that can beloaded in memory and executed by a processor included in theinfrastructure server 135, for example. The DNN 300 can include n inputnodes 305, each accepting a set of inputs i (i.e., each set of inputs ican include one or more inputs X). The DNN 300 can include m outputnodes (where m and n may be, but typically are not, a same naturalnumber) provide sets of outputs o₁ . . . o_(m). The DNN 300 includes aplurality of layers, including a number k of hidden layers, each layerincluding one or more nodes 305. The nodes 305 are sometimes referred toas artificial neurons 305, because they are designed to emulatebiological, e.g., human, neurons. The neuron block 310 illustratesinputs to and processing in an example artificial neuron 305 i. A set ofinputs X₁ . . . X_(r) to each neuron 305 are each multiplied byrespective weights w_(i1) . . . w_(ir), the weighted inputs then beingsummed in input function Σ to provide, possibly adjusted by a biasb_(i), net input a_(i), which is then provided to activation function ƒ,which in turn provides neuron 305 i output Y_(i). The activationfunction ƒ can be a variety of suitable functions, typically selectedbased on empirical analysis. As illustrated by the arrows in FIG. 3 ,neuron 305 outputs can then be provided for inclusion in a set of inputsto one or more neurons 305 in a next layer.

The DNN 300 can be trained to accept as input data, e.g., referenceimages from a camera, and to output one or more parameters foridentifying the meteorological optical range r. For example, the DNN 300could be trained to output a distance to actual fog.

That is, the DNN 300 can be trained with ground truth data, i.e., dataabout a real-world condition or state. Weights w can be initialized byusing a Gaussian distribution, for example, and a bias b for each node305 can be set to zero. Training the DNN 300 can including updatingweights and biases via conventional techniques such as back-propagationwith optimizations.

A set of weights w for a node 305 together are a weight vector for thenode 305. Weight vectors for respective nodes 305 in a same layer of theDNN 300 can be combined to form a weight matrix for the layer. Biasvalues b for respective nodes 305 in a same layer of the DNN 300 can becombined to form a bias vector for the layer. The weight matrix for eachlayer and bias vector for each layer can then be used in the trained DNN300.

In the present context, the ground truth data used to train the DNN 300could include image data annotated to identify the MOR r. For example, asensor can collect a plurality of images that can include simulated fog,as described above, and then can be labeled for training the DNN 300,i.e., tags can be specified identifying the MOR r, such as justdescribed, in the images. The DNN 300 can then be trained to output datavalues that correlate to the MOR r, and the output data values can becompared to the annotations to identify a difference, i.e., a costfunction of the output data values and the input annotated images. Theweights w and biases b can be adjusted to reduce the output of the costfunction, i.e., to minimize the difference between the output datavalues and the input annotated images. When the cost function isminimized, the server 125 can determine that the DNN 300 is trained.

FIG. 4 is a diagram of an example process 400 for training a machinelearning program for a vehicle 101. The process 400 begins in a block405, in which a server 125 receives an image 200 of a roadway. The image200 can be collected by, e.g., a camera 110 on a vehicle 101 on theroadway. The image 200 can be a red-blue-green (RBG) color image.Alternatively, the image 200 can be a black-and-white (BW) image.

Next, in a block 410, the server 125 applies simulated fog 205 to theimage 200. The simulated fog 205 is a change in color values of theimage 200 to mimic the occlusion caused by actual fog. The server 215can apply the simulated fog 205 with a color changing model, asdescribed above, that changes color values of pixels of the image 200.The color changing model can change the color values of the pixels basedon a specified meteorological optical range (MOR) r, i.e., a distancefrom a distance at which a luminosity of a beam of light extending froman emitter (e.g., a light source at the sensor 110) falls below aluminosity threshold. The luminosity threshold is a predetermined valueset by a standard setting organization, e.g., the International CivilAviation Organization. For example, the luminosity threshold can be 5%.

Next, in a block 415, the server 125 can annotate the image 200 with theMOR r of the simulated fog 205. That is, for each pixel in which theserver 125 includes simulated fog 205, the server 125 can provide alabel indicating the MOR r used to input the simulated fog 205 to thepixel.

Next, in a block 420, the server 125 inputs the images 200 with thesimulated fog 205 to a machine learning program 300 to output an MOR r.For example, the machine learning program can be a deep neural network300 trained to output the MOR r based on annotations of simulated fog205 in an image 200. When the machine learning program 300 is trained,the machine learning program 300 can output the MOR r of actual fog inan image 200 collected by a vehicle camera 110. The server 125 canadjust weights of nodes 305 of the DNN 300 to reduce output of a costfunction, i.e., the minimized the difference between output data valuesand the input annotated images 200.

Next, in a block 425, the server 125 determines whether the machinelearning program is trained. The server 125 can determine that themachine learning program is trained when the cost function is minimized.That is, the server 125 can determine that the cost function isminimized when inputting additional images 200 does not reduce output ofthe cost function further. If the server 125 determines that the machinelearning program is trained, the process 400 ends. Otherwise, theprocess 400 returns to the block 405 to receive another image 200.

FIG. 5 is a diagram of an example process 500 for operating a vehicle101 according to a machine learning program 300 trained as described inFIG. 4 . The process 500 begins in a block 505, in which a computer 105of the vehicle 101 receives an image 200 from a camera 110. As describedabove, the computer 105 can actuate the camera 110 to collect an image200 of a roadway in front of the vehicle 101. The image 200 can be,e.g., an RBG image 200.

Next, in a block 510, the computer 105 inputs the image 200 into themachine learning program 300. As described in FIG. 4 , the machinelearning program 300 can be a deep neural network 300 with a pluralityof layers trained to output a meteorological optical range (MOR) r offog in the image 200.

Next, in a block 515, the computer 105 identifies the MOR r of actualfog in the image 200. As described above, the machine learning program300, trained on images 200 with simulated fog 205, outputs MOR r valuesfor each pixel in the image 200. The computer 105 can identify a minimumMOR r in the image 200, that is, the smallest MOR r value output by themachine learning program 300.

Next, in a block 520, the computer 105 determines whether the minimumMOR r in the image 200 is below a threshold. The threshold can be aminimum distance for autonomous operation, i.e., have a minimum distancefrom the vehicle 101 from which data can be collected to operate acomponent 115 without manual input. For example, a brake can have aminimum distance of 60 meters because, upon actuating the brake, adistance to slow a vehicle 101 from a speed of 100 kilometers per hourto a stop is about 55 meters. Thus, the computer 105 can determine that,when the MOR r is less than 60 meters, the brake should be controlled bymanual input. If the MOR r is below the threshold, the process 500continues in a block 525. Otherwise, the process 500 continues in ablock 530.

In the block 525, the computer 105 transitions one or more components115 from autonomous operation to manual operation. As described above,the threshold is based on respective minimum distances to fog that thecomponents 115 can operate. For example, if the MOR r is 45 meters,which is greater than a minimum distance of 30 meters for a steering 120but less than a minimum distance of 60 meters for a brake 120, thecomputer 105 can transition the steering 120 to manual operation andmaintain autonomous operation of the brake 120.

In the block 530, the computer 105 determines whether to continue theprocess 500. For example, the computer 105 can determine to continue theprocess 500 until arriving at a destination and powering down thevehicle 101. If the computer 105 determines to continue, the process 500returns to the block 505 to collect another image 200. Otherwise, theprocess 500 ends.

Computing devices discussed herein, including the computer 105, includeprocessors and memories, the memories generally each includinginstructions executable by one or more computing devices such as thoseidentified above, and for carrying out blocks or steps of processesdescribed above. Computer executable instructions may be compiled orinterpreted from computer programs created using a variety ofprogramming languages and/or technologies, including, withoutlimitation, and either alone or in combination, Java™, C, C++, VisualBasic, Java Script, Python, Perl, HTML, etc. In general, a processor(e.g., a microprocessor) receives instructions, e.g., from a memory, acomputer readable medium, etc., and executes these instructions, therebyperforming one or more processes, including one or more of the processesdescribed herein. Such instructions and other data may be stored andtransmitted using a variety of computer readable media. A file in thecomputer 105 is generally a collection of data stored on a computerreadable medium, such as a storage medium, a random access memory, etc.

A computer readable medium includes any medium that participates inproviding data (e.g., instructions), which may be read by a computer.Such a medium may take many forms, including, but not limited to, nonvolatile media, volatile media, etc. Non volatile media include, forexample, optical or magnetic disks and other persistent memory. Volatilemedia include dynamic random access memory (DRAM), which typicallyconstitutes a main memory. Common forms of computer readable mediainclude, for example, a floppy disk, a flexible disk, hard disk,magnetic tape, any other magnetic medium, a CD ROM, DVD, any otheroptical medium, punch cards, paper tape, any other physical medium withpatterns of holes, a RAM, a PROM, an EPROM, a FLASH EEPROM, any othermemory chip or cartridge, or any other medium from which a computer canread.

With regard to the media, processes, systems, methods, etc. describedherein, it should be understood that, although the steps of suchprocesses, etc. have been described as occurring according to a certainordered sequence, such processes could be practiced with the describedsteps performed in an order other than the order described herein. Itfurther should be understood that certain steps could be performedsimultaneously, that other steps could be added, or that certain stepsdescribed herein could be omitted. For example, in the process 400, oneor more of the steps could be omitted, or the steps could be executed ina different order than shown in FIG. 4 . In other words, thedescriptions of systems and/or processes herein are provided for thepurpose of illustrating certain embodiments and should in no way beconstrued so as to limit the disclosed subject matter.

Accordingly, it is to be understood that the present disclosure,including the above description and the accompanying figures and belowclaims, is intended to be illustrative and not restrictive. Manyembodiments and applications other than the examples provided would beapparent to those of skill in the art upon reading the abovedescription. The scope of the invention should be determined, not withreference to the above description, but should instead be determinedwith reference to claims appended hereto and/or included in anon-provisional patent application based hereon, along with the fullscope of equivalents to which such claims are entitled. It isanticipated and intended that future developments will occur in the artsdiscussed herein, and that the disclosed systems and methods will beincorporated into such future embodiments. In sum, it should beunderstood that the disclosed subject matter is capable of modificationand variation.

The article “a” modifying a noun should be understood as meaning one ormore unless stated otherwise, or context requires otherwise. The phrase“based on” encompasses being partly or entirely based on.

What is claimed is:
 1. A system, comprising: a computer including aprocessor and a memory, the memory storing instructions executable bythe processor to: collect an image with a second vehicle sensor; inputthe image to a machine learning program trained by generating asynthetic image by adjusting respective color values of one or morepixels of a reference image based on a specified meteorological opticalrange from a first vehicle sensor to simulated fog, inputting thesynthetic image to theft machine learning program to train the machinelearning program to identify a meteorological optical range from thefirst vehicle sensor to actual fog; output, from the machine learningprogram, the meteorological optical range from the second vehicle sensorto the actual fog; and actuate one or more vehicle components based onthe meteorological optical range from the vehicle sensor to the actualfog, wherein the instructions further include instructions to actuate afirst component when the meteorological optical range from the secondvehicle sensor to the actual fog is above a first threshold for thefirst component to operate without manual input, and to actuate a secondcomponent when the meteorological optical range from the second vehiclesensor to the actual fog is above a second threshold for the secondcomponent to operate without manual input.
 2. The system of claim 1,wherein the instructions further include instructions to transition oneof the one or more vehicle components from autonomous operation tomanual operation when the meteorological optical range from the vehiclesensor to the actual fog is below a distance threshold.
 3. The system ofclaim 2, wherein the instructions further include instructions totransition a vehicle from an autonomous mode to a manual mode when themeteorological optical range from the second vehicle sensor to theactual fog is below a second distance threshold.
 4. The system of claim1, wherein the instructions further include instructions to output, fromthe machine learning program, an object in the collected image and toactuate the one or more vehicle components based on the object output bythe machine learning program.
 5. The system of claim 1, wherein theinstructions further include instructions to identify an ambient lightof the reference image and to adjust the respective color values of theone or more pixels based on the ambient light.
 6. The system of claim 1,wherein training the machine learning program includes annotating thesynthetic image with the meteorological optical range from the sensor tothe simulated fog.
 7. The system of claim 1, wherein training themachine learning program includes adjusting the color values of one ofthe pixels in the reference image by decreasing the color values of theone of the pixels based on a transmission coefficient of light throughfog and increasing an ambient light of the pixel based on thetransmission coefficient.
 8. The system of claim 1, wherein training themachine learning program includes identifying an initial distance fromthe first vehicle sensor to an object in one of the pixels of thereference image and adjusting the color values of the one of the pixelsbased on the initial distance and the specified meteorological opticalrange from the vehicle sensor to the simulated fog.
 9. The system ofclaim 1, wherein the instructions further include instructions toidentify an object with another second vehicle sensor and a distancefrom the other second vehicle sensor to the object, and to suppressactuation of one or more vehicle components based on the identifiedobject when the distance from the other second vehicle sensor to theobject exceeds the meteorological optical range from the second vehiclesensor to the actual fog.
 10. The system of claim 1, wherein themeteorological optical range is a distance at which a luminosity of abeam of light extending from the vehicle sensor falls below a luminositythreshold.
 11. A method, comprising: generating a synthetic image byadjusting respective color values of one or more pixels of a referenceimage based on a specified meteorological optical range from a firstvehicle sensor to simulated fog, the meteorological optical range beinga distance at which a luminosity of a beam of light extending from thevehicle sensor falls below a luminosity threshold; inputting thesynthetic image to a machine learning program to train the machinelearning program to identify a meteorological optical range from thefirst vehicle sensor to actual fog; collecting an image with a secondvehicle sensor; inputting the image to the machine learning program;outputting, from the machine learning program, the meteorologicaloptical range from the vehicle sensor to the actual fog; and actuatingone or more second vehicle components based on the meteorologicaloptical range from the vehicle sensor to the actual fog, includingactuating a first component when the meteorological optical range fromthe vehicle sensor to the actual fog is above first threshold for thefirst component to operate without manual input, and to actuate a secondcomponent when the meteorological optical range from the second vehiclesensor to the actual fog is above a second threshold for the secondcomponent to operate without manual input.
 12. The method of claim 11,further comprising transitioning one of the one or more second vehiclecomponents from autonomous operation to manual operation when themeteorological optical range from the vehicle sensor to the actual fogis below a distance threshold.
 13. The method of claim 11, furthercomprising outputting, from the machine learning program, an object inthe collected image and actuating the one or more second vehiclecomponents based on the object output by the machine learning program.14. A system, comprising: a vehicle sensor; means for generating asynthetic image by adjusting respective color values of one or morepixels of a reference image based on a specified meteorological opticalrange from the vehicle sensor to simulated fog, the meteorologicaloptical range being a distance at which a luminosity of a beam of lightextending from the vehicle sensor falls below a luminosity threshold;means for inputting the synthetic image to a machine learning program totrain the machine learning program to identify a meteorological opticalrange from the vehicle sensor to actual fog; means for collecting animage with the vehicle sensor; means for inputting the image to themachine learning program; means for outputting, from the machinelearning program, the meteorological optical range from the vehiclesensor to the actual fog; and means for actuating one or more vehiclecomponents based on the meteorological optical range from the vehiclesensor to the actual fog, including actuating a first component when themeteorological optical range from the vehicle sensor to the actual fogis above a first threshold for the first component to operate withoutmanual input, and to actuate a second component when the meteorologicaloptical range from the second vehicle sensor to the actual fog is abovea second threshold for the second component to operate without manualinput.
 15. The system of claim 14, further comprising means fortransitioning one of the one or more vehicle components from autonomousoperation to manual operation when the meteorological optical range fromthe vehicle sensor to the actual fog is below a distance threshold. 16.The system of claim 14, further comprising means for outputting, fromthe machine learning program, an object in the collected image and meansfor actuating the one or more vehicle components based on the objectoutput by the machine learning program.